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ABSTRACT 



A continuing and vexing problem associated with survey 
instrument development is the creation of items, initially, that correlate 
favorably a posteriori with constructs being measured. This study tests the 
use of symbolic- logic matrices developed by D. G. Coleman (1979) in creating 
factorially "pure" statistically discrete constructs in survey instrument 
design. The study used symbolic- logic (verbal logic) "truth tables" to 
increase statistical factor loadings in confirmatory factor analysis studies 
and to develop discrete constructs. Initial experimental work with truth 
tables increased, hypothetically, construct purity in construct validation. 
Construct validation helps to assure that items created for measuring 
specific theoretical or real constructs do, indeed, represent the distinct 
qualities measured. In November 1996, the National Association of Elementary 
School Principals (NAESP) approved the revision of the Administrative 
Diagnostic Inventory (ADI) , the title given to the diagnostic procedure 
conducted within their assessment center. To assist the NAESP in the use of 
the ADI for the assessment of current and prospective school principals, this 
study evaluated the construct validity of the instrument with 203 teachers 
enrolled in educational administration programs. Results show the improved 
content validity and construct validity of the revised ADI. Factor analysis 
shows that the descriptors within the skill dimensions are highly correlated 
and are significant measures of the constructs for which they were written. 
(Contains 2 tables and 20 references.) (SLD) 
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Abstract 



A continuing and vexing problem associated with survey instrument development is 
the creation of items, initially, that correlate favorably a posteriori with constructs being 
measured. The purpose of the study tests the use of symbolic-logic matrices developed by 
Coleman (1979) in creating factorily “pure” statistically discrete constructs in survey 
instrument design. 

The study used symbolic-logic (verbal logic) “truth tables” to increase statistical 
factor loadings in confirmatory factor analysis studies and to develop discrete constructs. 
Initial experimental work with truth tables increased, hypothetically, construct purity in 
construct validation. Construct validation helps to assure that items created for measuring 
specific theoretical or real constructs do, indeed, represent the distinct qualities measured. 

In November 1996, the National Association of Elementary School Principals 
(NAESP) approved the revision of the Administrative Diagnostic Inventory (ADI), the title 
given to the diagnostic procedure conducted within their assessment center. To assist the 
NAESP in the use of the ADI for the assessment of current and prospective school 
principals, this study evaluated the construct validity of the instrument. 
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USING SYMBOLIC MATRICES TO IMPROVE 
CONFIRMATORY FACTOR ANALYSIS TECHNIQUES 

INTRODUCTION 

The purpose of the study was to validate the constructs of the revised Administrative 
Diagnostic Inventory (ADI), since no validation studies existed. The validation was necessary to 
assure that scores achieved on individual items (descriptors) within each construct correlate 
favorably with one another on the 13 constructs and that the newly re-designed constructs 
maintain individual identities. The ultimate goal of the study was to “confirm” the discreteness 
of the 13 behavioral constructs first identified and the intercorrelation of the eight descriptors 
identified within each construct. In other words, the study sought to determine whether the 13 
constructs measure separate and distinct characteristics of effective school principals and 
whether the eight descriptors developed for each construct are intercorrelated and measure 
specific constructs. 

Data for the study originated from a paper and pencil instrument administered to teachers 
and entry level administrators who evaluated their supervising principal. The protocol used in 
the construct validity required subjecting these survey data to confirmatory factor analysis, using 
R factor analysis with Kaiser normalization and Oblimin rotation. 

With the commitment of the National Association of Elementary School Principals 
(NAESP) to use the ADI for the assessment of current and prospective principals, construct 
validity is crucial and essential. The results of the study will assist the NAESP and assessment 
centers across the country with the goal of using valid measures to assess educational 
administrators. 
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THEORY/RATIONALE 

Research indicates that the school principal is recognized as a significant factor in school 
success. The quality of teaching and student learning, the school climate, and the confidence in 
public schools are directly related to the performance of school principals (Lipman, 1981; Smith 
& Andrews, 1989). The importance of school principals necessitates a need for effective ways 
to evaluate and assess school administrators. The field’s conceptualization of organizational 
processes, including the leadership construct, is constantly evolving (Glasman & Heck, 1992; 
Hallinger, 1992; Leithwood & Hallinger, 1993). The assessment of leadership and management 
skills and abilities is now recognized as a potential tool in the preparation, development, 
selection, and professional growth of school principals (Edmonds, 1982). 

One method for identifying and measuring administrative potential is the assessment 
center, a model created with simulations and exercises to assess skills and behaviors of individuals 
(Elsaesser, 1990). The field of education developed performance assessment systems to address 
the heightened concerns for schooling and for finding good administrators (Sirotnik & Durden, 
1996). The creation of the ADI was in response to the demands for principal effectiveness 
substantiated by stringent accountability measures. 

Showing the relationship between variables and constructs represents a very important 
part of the content of any scientific field (Comrey, 1973). In some fields, such as education, less 
agreement exists among researchers concerning what variables relate to each other. The 
relationships between variables are often not clearly defined. Factor analysis represents a 
growing body of statistical methods that can be of great value in these areas (Hair, 1979). Factor 
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analytic methods help educational researchers to define their variables and relate them to each 
other in an attempt to develop their science to a higher level. 

A research and writing team of eight professional educators addressed the content validity 
of the revised ADI in late 1995 and early 1996 (Coleman & Creighton, 1996). By verifying the 
construct validity of the instrument, districts will gain in planning relevant and productive staff 
development for school principals. In addition, educational leadership institutions can improve 
the development of administrative training programs. 

The purpose of the study was to use confirmatory factoring procedures to conduct a 
construct validation of the ADI. The research questions derived from the rationale and used to 
focus the study were: 

1 . Are the eight descriptors within each of the 13 constructs intercorrelated and truly 
measuring the specific construct? 

2. Are the 13 constructs on the ADI discrete and separate measures of effective 
elementary school principals? 

NEED FOR ADI REVISION 

Validity studies of the ADI (Dale, 1988; Menrad, 1990; Elsaesser, 1990; James, 1991; 
Durden, 1994; Sirotnik & Durden, 1996) emphasized a need for construct validation. Though 
research indicated limited data on various elements of the ADI in dissertations, no construct 
validation studies were completed (Coleman, 1995). 

California State University Fresno (CSUF) was the first franchise signed by the National 
Association of Elementary School Principals (NAESP) to use the ADI after the rights had been 
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purchased from the University of Washington. By October 1995, CSUF had conducted 13 
assessments covering approximately 150 candidates (Coleman, 1995). 

From the beginning, the validity and reliability of the ADI was questioned. Though the 
ADI was believed to have been created around sound educational theory, and by a wide variety 
of educational experts and practitioners, research from the field raised concerns regarding 
construct validity. 

In November 1995, Coleman prepared a report to the NAESP detailing three construct 
validation studies. The first one by Petrulis (1994) and the other by Coleman (1994) produced 
similar results: constructs were very difficult to decipher. 

The third study (Coleman, 1995) subjected each of the 12 constructs to individual 
construct validity measures with equally disappointing results. Thirty-seven constructs were 
found instead of the purported 12 on the ADI instrument. Descriptors failed to load on the 
appropriate constructs and little variance was accounted for by the first six factors (31.9%). The 
evidence supported revising the ADI and completing additional studies of the instrument 
authenticating its validity. 

A major concern pertained to evidence that the definition stems used in defining the 
constructs appeared as multiple rather than singular measures. Each of the constructs therefore, 
displayed a variety of phrases supposedly describing one specific construct. This resulted in the 
construct measuring several rather than a single variable (Coleman, 1995). If multiple stems are 
used, as in the case of the ADI, the data may sort into multiple factors on a single construct as 
indicated by many factors showing up in a factor analysis. 
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The Coleman study indicated that the content validity of the ADI was not a major 

problem, but indicated that the constructs did not necessarily measure the whole range of 

principal effectiveness variables. Some of the constructs were not mentioned in contemporary 

literature as significant in principal effectiveness and others appeared to overlap each other. 

In discussing the earlier ADI, Sirotnik and Durden (1996) pointed to the issue of 

construct validity and suggested that the 12 ADI dimensions did not appear to be independent 

measures of leadership as presently defined and operationalized. They contended that the faulty 

dimensions may be improved by including more involvement from other educator groups (such as 

teachers and school-based administrators) in the refinement and development of the ADI. 

Durden’s recent research on the ADI (1994) had emphasized the importance of regular and 

constant evaluation and revision of assessment center models of performance evaluation: 

Leadership expectations are changing. Analysis of assessment centers should be 
regular and updated periodically, every four years or so. This is another reason for 
further research of the ADI and NASSP systems, since the original job analysis was 
conducted more than five years ago. (p. 252) 



FACTOR ANALYSIS 

The use of factor analysis to provide construct validation is not new. In Educational and 

Psychological Measurement, Guilford (1946) stated: 

The factorial validity of a test is given by its loadings in meaningful, common, reference 
factors. This is the kind of validity that is really meant when the question is asked “Does 
this test measure what it is supposed to measure?” (pp. 437-438) 
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Factor analysis is used in this study to check for discreteness among constructs and 
intercorrelations among items used in measurement. The validation procedure provides additional 
assurance that what is being measured is valid. 

Factor analysis has two primary functions in data analysis (Nunnally, 1978). One 
function is to identify underlying constructs in the data. As an example, the analysis may show 
the variables of “seeks clarity in purpose” and “involves stakeholders in the planning process” to 
actually be indicators of the same theoretical construct (e.g., planning). A second role of factor 
analysis is simply to reduce the number of variables to a more manageable set. 

In reducing the number of variables, factor analysis procedures attempt to retain as much 
information as possible and to make the remaining variables as meaningful and as easy to work 
with as possible. The general purpose of factor analytic techniques is to condense the 
information contained in a number of original variables into a smaller set of composite factors, 
with minimum loss of information. 

METHODOLOGY 

In November 1995, the National Association of Elementary School Principals (NAESP) 
decided to revise the ADI assessment instrument. Education administration professors from five 
universities, practicing school superintendents and principals, research consultants, assessment 
center directors, and representatives from NAESP began the process immediately, completing the 
new instrument in February 1996 (Creighton, 1996). 
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Step 1 

The initial assignment of the writing and research team was to identify constructs believed 
important to school administrators. The team began by researching constructs and descriptors 
used by other organizations, various national accreditation agencies, task forces, and other 
assessment instruments. Previous ADI assessment instruments, National Association of 
Seconday School Principals (NASSP) assessment dimensions, National Council for the 
Accreditation of Teacher Education (NCATE) standards, National Policy Board for Educational 
Administration (NPBEA) competencies, and a literature review were studied. The purpose of 
examining various recommended competencies was to identify and select the most appropriate 
and applicable constructs to be used for the ADI. 

The writing team analyzed the existing models of proficiencies, standards, domains, and 
dimensions to form appropriate constructs for the ADI. The team first examined existing 
constructs individually, and then collectively, to determine those believed to be most important. 
The result was an agreement on 1 3 constructs (dimensions), or knowledge and skills of effective 
school leaders. These skills covered both management (personal skills) and leadership (inter- 
personal) skills. The constructs are listed in Figure 1, along with the 12 original ADI constructs 
for comparison. 

Step 2 

The next step was to define the newly identified constructs. The writing team studied 




previously written definitions from the competency and standards documents. The team 
maintained that many of the original definitions were too broad and contained several stems that 
would cause overlap with other constructs and definitions, resulting in weak validity and 
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reliability. The creation of single stem definitions resulted in more clear and concise definitions. 
An example is provided in Figure 2, comparing two definitions for creativity and identifying 
multiple vs single stem definitions. Previous research (Andrews, 1965; Brubaker, 1983; Coleman, 
1995) indicates that content validity is decreased with such multiple stem constructs. 



REVISIED ADI 


PREVIOUS ADI 


1 . 


Planning 


Problem Analysis 


2. 


Organizing 


Judgement 


3. 


Problem Solving 


Organization Ability 


4. 


Decisiveness 


Decisiveness 


5. 


Creativity 


Group Leadership 


6. 


Systems Analysis 


Instructional Leadership 


7. 


Vision 


Creativity 


8. 


Communications 


Oral Communications 


9. 


Instructional Leadership 


Written Communication 


10. 


Group Leadership/Team Building 


Human Relations 


11. 


Climate Development 


Resourcefulness 


12. 


Moral Responsibility 


Educational Values 


13. 


Instructional Analysis 





FIGURE 1 

ADI Construct Comparison 
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CREATIVITY (original ADI) 

Generates and recognizes innovative solutions in work-related situations; exhibits openness 
to new ideas from others; demonstrates originality in developing policies and procedures; 
performs under pressure and during opposition; thinks well during oral interchange; 
maintains effectiveness in varying environments with various tasks, responsibilities, 
and people; exhibits behavior flexibility; adjusts time schedule as need arises; and has 
tolerance for ambiguity. 



CREATIVITY (revised ADI) 

Demonstrates innovation and inventiveness in work-related situations. 



FIGURE 2 

Multiple vs Single Stem Definitions 



Step 3 

After reaching agreement on 1 3 construct definitions, the team members individually 
wrote “descriptors” that would provide differing measures for each construct. When finished, 
they reviewed the list to select the most relevant. The task was complex and required a great deal 
of reworking the items. As an example, the team agreed upon a descriptor for creativity, open 
and receptive to nontraditional sources of information. After considerable debate and discussion, 
a team member asked how “open and receptive” could be observed in principal behavior? The 
team agreed with the concern and rewrote the descriptor to read: demonstrates openness and 
receptivity to nontraditional sources of information.. The team agreed that, under certain 
circumstances, the act of demonstrating openness and receptivity could be observed in the 
behavior of a principal. The issue for the team was to select descriptors that were specific, 
interrelated with each other, and measured the construct. 
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Step 4 

To improve the inner-item correlations and the construct validity of the constructs, the 
team used symbolic-logic matrices to verify that each descriptor logically agrees with others 
measuring the same construct. A symbolic-logic matrix resembles a correlation matrix but allows 
individual items to be compared according to verbal construction (logic) by subjecting them to 
“if/then” statements (Coleman, 1993 and 1995). 

When using symbolic-logic matrices to compare dimensions or constructs, two-way 
disagreements are desired since constructs should be discrete and measure different behaviors. 
Figure 3 demonstrates “if/then” statements for instructional leadership and creativity. If the two 
constructs are truly discrete and separate measures of effective school leadership, the “if/then” 
statements should disagree logically. The disagreement should produce a two-way disagreement, 
which means the reversed “if/then” statement should also result in a “no, or not necessarily.” A 
principal who works effectively with the school community for example, does not necessarily 
demonstrate innovation and inventiveness. Reversing the statement also results in a negative 
consequence. The symbolic-logic matrix can be used to identify discreteness and to help assure 
differences between constructs. 

Figure 4 illustrates a symbolic-logic matrix with the construct created for instructional 
analysis and the eight matching descriptors. When using the matrix to strengthen correlation 
between descriptors, two-way agreements are desired. Two-way agreement best predicts high 
inter-item correlation. The intent of using the matrices was to refine and tighten the constructs 
and descriptors on the ADI. 
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INSTRUCTIONAL LEADERSHIP 

works effectively with the school community to advance student learning. 

CREATIVITY 

demonstrates innovation and inventiveness in work-related situations. 

If/then statements: 

1. If a principal works effectively with the school community to advance student learning, then he/she 
demonstrates innovation and inventiveness in work-related situations. 

2. If a principal demonstrates innovation and inventiveness in work-related situations, then he/she works 
effectively with the school community to advance student learning. 



FIGURE 3 

Creating a Symbolic- Logic Matrix with Two ADI Constructs 



Pointing out a distinction between two-way agreements and two-way disagreements may 
help clarify the appropriate use of the symbolic-logic matrices. In using the matrices to improve 
the intercorrelation of items, as in the case of the ADI descriptors, two-way agreements are 
desired. Two-way agreement between and among items across the entire matrix measuring a single 
construct should reflect reasonably high inter-item correlation. However, when the desire is to 
maintain discreteness, as in the case of the ADI constructs, two-way disagreements are 
necessary. To retain discrete constructs in a multi-construct design, care must be maintained to 
create constructs that are independent of each other and possessing items reflecting low levels of 
statistical correlation with items from other constructs (Coleman, 1996). The symbolic-logic 
matrices should reflect two-way disagreement among the constructs when low level of inter-item 
correlation is desired. 




14 



12 



INSTRUCTIONAL ANALYSIS: works effectively with teachers to improve instruction. 



Variables 


1 


2 


3 


4 


5 


6 


7 


8 


1. encourages instructional 
improvement 


XX 


A 


A 




A 


A 


A 


A 


2. provides suggestions helpful to 
teachers for improving instruction 


A 


XX 


A 


.R 


A 


.R 


A 


A 


3. possesses high expectation for 
student improvement 


A 




XX 


A 


A 


A 


A 


A 


4. demonstrates interest in student 
intellectual growth and development 




.R 




XX 


A 




A 


A 


5. emphasizes student achievement 


A 




A 


A 


XX 


A 


A 


A 


6. demonstrates expertise in analyzing 
teaching and learning 


A 


.R 


j A 


A 


A 


XX 


A 


A 


7. maintains expertise of alternative 
instructional practices 


A 




A 


A 


A 


A 


XX 


A 


8. possesses knowledge of activities 
occurring in various classrooms 


A 




A 


A 


A 


A 


A 


XX 



FIGURE 4 

Completed Symbolic-Logic Matrix with ADI Construct and Descriptors 



Note 



The numbers across the top of the matrix represent the eight descriptors, and the “XX” are placed in the cell 
where the descriptor correlates with itself. Obviously, the descriptor correlated with itself would result in a 
1.00 correlation. The purpose of this process is to verbally correlate the descriptors with each other and the 
construct. 

Starting with a correlation between descriptor 1 and descriptor 2 (second cell across the first row), an “if/then” 
statement is created to read, “if a principal encourages instructional improvement (descriptor 1), then he/she 
provides suggestions helpful to teachers for improving instruction (descriptor 2).” If the statement is agreed to, 
an “A” is placed in the cell representing agreement. If the statement is not agreed to, a “dot” is placed in the 
cell representing one-way disagreement. The evaluator moves on to the third cell across the first row, until 
each item serves as the consequence to the original item serving as the antecedent. 

As the evaluator begins to address the columns, a need exists to reverse the “if/then” statement. For instance, 

1 across correlated with 2 down is the reverse “iffthen” statement as 2 across correlated with 1 down. If a 
two-way disagreement is encountered, an “R” (rejection) is placed in the cell. A “dot” indicates one-way 
disagreement; an “R” indicates two-way disagreement; and an “A” represents two-way agreement. 

A two-way agreement has the best chance of having high inter-item correlation. A one-way agreement drops 
the probability of a high correlation coefficient while two-way rejection drops the coefficient even further. 
Descriptors which have two-way disagreements recorded (R), are considered poor and either reworked 
or discarded. 
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DATA COLLECTION: SURVEY/QUESTIONNAIRE 

A paper and pencil survey instrument with the 1 04 ADI descriptors provided for 
collecting information on practicing school administrators and for verifying the construct validity 
of the ADI. The intent was to further purify the descriptors and constructs and collect data 
essential for use in a field-based version of the ADI. Eight items were created for each construct 
to provide for selecting at least six highly inter-correlated items. Methodically placing the 104 
descriptors throughout the survey prevented the respondents from recognizing any of the 1 3 
individual constructs. The intent was to provide each respondent the opportunity to answer 
each item independently while minimizing bias or preconceived notions. Survey respondents 
rated their current administrator with the 104 descriptors on a 6-point Likert scale. Testing the 
relationships between descriptors and among constructs was one of the purposes of the survey. 

Two-hundred-three teachers enrolled in the educational administration programs at 
California State University Fresno, and teachers and vice-principals in the California School 
Leadership Academy completed the survey during February and March 1996. Formatting the 
responses on diskette with Microsoft Excel 4.0 allowed for the transfer of data to the Statistical 
Package for the Social Sciences (SPSS v6.1). The study subjected the data to R factor analysis 
with Kaiser normalization and Oblimin rotation. By subjecting the data from the 203 survey 
respondents to factor analysis, the intent was to confirm the correlations between individual 
constructs and descriptors, and to address the two research questions developed for study 
direction and focus. 

The use of factor analysis had two functions in the study. The first was to identify 
underlying constructs in the data, the second, to reduce the number of variables to a more 
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manageable set. In reducing the number of variables, factor analysis procedures attempt to retain 
as much of the information as possible and to make the remaining variables as meaningful and as 
easy to work with as possible. The factor rotation scheme selected in the study was Oblimin 
oblique to maximize the interpretation of factor loadings. 

FINDINGS AND INTERPRETATIONS 

Addressing the first research question involved running a correlation matrix for each of the 
13 constructs with the accompanying eight descriptors. Analyzing the resulting data determined 
the extent of correlation among descriptors and the degree to which the descriptors represented a 
single factor. Two or more factors representing the eight descriptors indicated more than one 
construct underlying the descriptors. 

Correlation matrices for the 13 ADI constructs revealed strong intercorrelation among the 
items for 1 1 of the constructs. The correlation coefficients were well above the suggested 
significance level of .18, with most ranging from .60 to .80, indicating a high degree of correlation 
among descriptors. As an example, the ADI construct for creativity is shown in Table 1. The 
data reveal all coefficients above . 1 8 and only one extracted factor possessing an eigenvalue of 
5.47. Factor loadings for the descriptors ranged from .73 to .91. The analysis suggests the items 
are effective measures of the construct, and identify a single factor as predicted (Creighton, 

1996). In addition, the eight descriptors for 1 1 of the constructs loaded on one specific factor 
(ranging from .70 to .90), further suggesting the items as clear measures of one skill dimension. 
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TABLE 1 







Correlation Matrix 


- ADI Construct of CREATIVITY 








V004 


V017 


VO30 


V043 V056 


V068 V081 


V095 


VO04 


1.00 














V017 


.77 


1.00 












VO30 


.73 


.69 


1.00 










V043 


.66 


.72 


.68 


1.00 








V056 


.77 


.76 


.73 


.71 1.00 








V068 


.63 


.67 


.55 


.59 .77 


1.00 






V081 


.61 


.50 


.63 


.53 .63 


.52 


1.00 




V095 


.50 


.47 


.47 


.64 .60 


.57 


.66 


1.00 


Variable Communality 


Factor 


Eigenvalue 


Percent of Variance 


Variable 


Factor 1 


VO04 


.75 


1 


5.47 


68.3 




VO04 


.86 


V017 


.85 


2 


.73 


9.1 




V017 


.85 


VO30 


.83 


3 


.53 


6.7 




VO30 


.83 


V043 


.84 


4 


.44 


5.5 




V043 


.84 


V056 


.91 


5 


.27 


3.4 




V056 


.91 


V068 


.80 


6 


.21 


2.7 




V068 


.80 


V081 


.76 


7 


.18 


2.3 




V081 


.76 


V095 


.73 


8 


.16 


2.1 




V095 


.73 



The communality tables further support the strength of the descriptors for the 1 1 
constructs. The high communality coefficients indicate that the factor accounts for most of the 
variance. In other words, the descriptors for the construct have much in common. 

A restatement of the first research question may be helpful: Are the eight descriptors for 
each of the 13 constructs intercorrelated and measuring the specified construct? The correlation 
matrices for the ADI constructs revealed the descriptors for 1 1 of them to be correlated from .47 
to .96. These 1 1 constructs also loaded on a single factor each. Clearly, 1 1 of the 13 constructs 
are unidimensional. 

Two constructs, climate development and instructional analysis, however, resulted in two 
factors identified by eigenvalues greater than 1.00. The rotated factor matrix for these two 
factors showed some variables loading highly on one factor and other variables loading highly on 
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the second factor. This splitting of the construct over two factors points to ambiguity in 
responses to the eight descriptors for each. In other words, the eight descriptors are not 
measuring a discrete construct. 

Factor Analysis 

The first step in addressing the second research question involved running an unrotated 
factor analysis, or principal components method, of the 104 descriptors (variables) to identify 
the sources of variance and factorial structure in the data. The result was the identification of 17 
factors with eigenvalues of 1 .0 and greater, four more than expected. Factors 1 8 through 1 04 are 
considered trivial and are not interpreted. 

The 17 factor solution was subjected to Oblimin (oblique) rotation to improve the 
interpretation. The major option to the analyst in rotation is to choose between an orthogonal 
method or an oblique method. An oblique factor rotation is desirable because it is theoretically 
and empirically more realistic (Hair, Anderson, & Tatham, 1979). 

The result was disappointing. Only six of the 13 ADI constructs loaded heavily on 
individual factors. The other seven constructs had so few descriptors clustering in groups that 
identifying or labeling them was difficult. 

The majority of significant descriptors (.50 or greater) loaded on 6 factors. Four factors 
noticeably identified discrete constructs as written for the ADI. Factor 1 consisted of seven of 
the eight descriptors written for the ADI construct problem solving which implies a label or name 
for Factor 1 as problem solving. Factor 2 contains seven of the eight descriptors written for the 
construct of creativity. Factor 8 contains seven of the eight descriptors written for the construct 
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of communication, and Factor 12 contained six of the eight descriptors written for the construct 
of organization. Factors 4 and 6 appeared somewhat ambiguous, but upon careful analysis, both 
factors contained heavy loadings for descriptors written for instructional analysis and 
instructional leadership. 

The descriptors for climate development and instructional analysis consisted of lower 
correlation coefficients, many correlating with descriptors for other constructs. The eight 
descriptors branched into separate areas, rather than measuring one discrete construct. 



DISCUSSION 

The use of symbolic-logic matrices by the ADI writing team resulted in the creation of 1 1 
constructs with fairly strong measurement items. For what reasons do the symbolic-logic 
matrices not have the same result with the constructs of climate development and instructional 
analysis? What causes these two constructs to split into two different factors? 

The two factors for climate development shown in Table 2 can be labeled (a) the psycho- 
socio environment of management and (b) the physical environment of management, reflecting the 
content of the descriptors. The one, maintains a facilitative psychological and social 
environment, implying attention to a psycho-socio detail. The other, attends to aspects of the 
physical environment, implying attention to physical details. The suggestion is not that the 
construct of climate development is an inappropriate construct forjudging effective school 
administrators, but as written, is comprised of two distinct concepts. Two possible remedies 
exist: (a) rewrite the descriptors to load on one or the other or (b) create two separate constructs, 
thus adding a construct to the ADI instrument. 
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TABLE 2 



Rotated Factor Matrix of ADI Construct Climate Development 



Descriptor 


Factor 1 


Factor 2 


VOll 


.36 


.61 


V024 


.67 


.09 


V037 


.11 


.84 


VO50 


.87 


.08 


V063 


.02 


.72 


V076 


.96 


.05 


V089 


.84 


.06 


VI 02 


.39 


.65 


Descriptor V037 






attends to aspects of the physical environment to maximize learning. 




Descriptor V076 


maintains a facilitative psychological and social environment. 





The same difficulty marked instructional analysis, which also divided into two factors. 
These factors were labeled (a) emphasizing student achievement and (b) emphasizing teaching. 
This construct can be improved in the same manner as suggested for climate development. 



RECOMMENDATIONS 

A primary function of factor analysis is to reduce redundancy and ambiguity. The data 
analysis of the study clearly identifies the ADI constructs of climate development and 
instructional analysis as being indiscrete measures. These two dimensions are important skill 
dimensions of effective school leadership and should not be discarded as constructs. Again, the 
suggestion is not that these two constructs are inappropriate skill dimensions of effective school 
principals, but that the constructs as written contain two underlying constructs within one. Both 
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constructs can be improved by rewriting descriptors to load on one rather than two factors. This 
could be accomplished by eliminating ambiguous and conflicting wording. 

Looking at the correlation matrix and communality table for each of the 1 3 ADI 
constructs, the data reveal 1 or 2 descriptors with lower coefficients. Several of these descriptors 
also failed to load on any of the identified 17 factors on the oblique rotated factor matrix. 
Repeating a factor analysis with a reduced number of descriptors (variables) is recommended. 
Eliminating the two lowest coefficients in each construct would reduce the variables from 104 to 
78. The correlations between variables would likely improve and assist in further improving the 
construct validity of the ADI. 

The ADI writing team was careful to subject the eight descriptors within each construct 
to the symbolic-logic matrix process. The team also utilized a 13 X 13 symbolic-logic matrix to 
test the 13 constructs for discreteness. The team failed however, to go further and create a 104 X 
104 matrix to test the eight descriptors within each construct with the descriptors of other 
constructs for discreteness. In other words, testing the construct definitions for disagreement on 
the symbolic-logic matrix does not assure the descriptors within each construct to be discrete 
when compared to descriptors within other constructs. 

The symbolic-logic matrix process which was so successful with the creation of 
descriptors within constructs can be of further help with the issue of construct correlations. The 
same process used for two-way agreement of descriptors can be equally valuable in seeking two- 
way disagreements. Though the constructs were worked through this process by the ADI 
writing team, the individual descriptors across constructs were not. 
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Subjecting the descriptors to a 104 X 104 symbolic-logic matrix is suggested for 
improving the discreteness of ADI constructs. Though the process would be difficult for one 
person, teams or groups could work through the process in a 'reasonable amount of time. If two- 
way disagreements could not be reached, individual descriptors would need to be reworded or 
modified in some way. Using a 104 X 104 matrix, descriptors within one construct can be 
compared with descriptors within another construct. For example, a descriptor within the 
construct of problem solving compared with a descriptor in the construct of planning needs to 
result in a two-way disagreement. Though the correlation between decriptors within each 
construct should be high, the correlations across constructs should be low. 

An alternative recommendation for improving the discreteness of constructs involves the 
creation of a smaller and more manageable symbolic-logic matrix. The data suggest six of the 13 
constructs to be appropriately discrete. The creation of a matrix for the remaining seven would 
concentrate only on the ones appearing weak. This procedure would require a smaller truth table 
of 56 X 56. Even smaller 8X8 symbolic-logic matrices could be created to match existing high 
correlations from the factor matrix. For example, a high correlation exists between Factor 1 
(problem solving) and Factor 2 (creativity). Using an 8 X 8 matrix comparing the eight 
descriptors of problem solving with the eight descriptors of creativity would result in the 
tightening of construct discreteness. 

SUMMARY 

The revised Administrative Diagnostic Inventory (ADI) shows significant improvements 
in content validity and more specifically, construct validity of the instrument. Factor analysis 

erIc 
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reveals the descriptors within each of the 13 skill dimensions (constructs) to be highly correlated 
and significant measures of the construct for which they were written. 

Factor analysis also reveals six of the 13 constructs to be discrete and separate measures 
of effective behavior of school principals. The seven constructs which did not prove discrete can 
likely be improved by subjecting the 104 descriptors to the verbal logic of symbolic matrices. 

Even though the ADI contains some weaknesses, the instrument proves to be 
significantly improved over the previous model. Including so many practitioners from the field 
and professionals from several universities in the revision process can be credited with the 
successful improvement of the ADI. 

The problem of reliability in factor analysis is always an issue. Like any other statistical 
procedure, a factor analysis starts with a set of imperfect data. When the data change because of 
changes in the sample, the data gathering process, or the many kinds of measurement errors, the 
results of the analysis will change also. Therefore, the results of any single analysis are not 
entirely dependable. If the result of this study has been to encourage further research and 
analysis of the Administrative Diagnostic Inventory, an important purpose is realized. As the 
leadership construct constantly evolves, regular and ongoing validity studies are crucial and 
necessary. 
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